Peripheral component interconnect (PCI) is widely used as a bus standard connecting computing devices such as central processing units (CPUs), hard disks, and graphics controllers and networking devices such as forwarding engines and network controllers. As a next-generation standard of this PCI, PCI express, which uses serial buses, instead of parallel buses used in the PCI, and which executes communication in packet format, has been standardized.
Non-Patent Document 1 discloses a method of connecting a plurality of CPUs on a system including such PCI express switch. FIG. 1 illustrates a system configuration realized by Non-Patent Document 1, and FIGS. 2 and 3 illustrate configurations of MRA (multi-root aware) PCI express switches proposed by Non-Patent Document 1. As illustrated in FIG. 1, the system using the MRA PCI express switches requires a multi-root PCI manager software MR-PCIM 52 on one of the CPUs connected to the switches to manage configuration states of the switches.
Further, MR end points 505 and 506, which are I/O devices accommodating access from a plurality of root complexes, can be connected to the MRA PCI express switches. The multi-root PCI manager sets the MRA PCI express switches, so that a plurality of root complexes can be connected to each other and can access the MR end point devices. Configurations of the MRA PCI express switches and an outline of a system operation will be hereinafter described. FIG. 1 illustrates a system configuration using two MRA PCI express switches. FIGS. 2 and 3 illustrate internal configurations of MRA PCI express switches 511 and 512, respectively.
To accommodate a plurality of root complexes and an MR end point, the MRA PCI express switch 511 includes: PCI-PCI bridges 5111, 5112, 5122, and 5113 to 5115 different from a conventional PCI express switch; virtual PCI express switches 5116 to 5119 each executing switch processing between bridges; an MRA controller logic 5120 set and controlled by the MR-PCIM 52; and a setting register 5121 storing setting information about the MRA controller logic 5120.
Likewise, the MRA PCI express switch 512, which is another MRA PCI express switch used to extend the number of connection ports, includes: PCI-PCI bridges 5123, 5124, and 5133 different from a conventional PCI express switch; virtual PCI express switches 5126, 5127, and 5129 each executing switch processing between bridges; an MRA controller logic 5120 set and controlled by the MR-PCIM 52; and a setting register 5131 storing setting information about the MRA controller logic 5120.
As in a conventional PCI express, the MRA PCI express switch establishes a tree for each of the plurality of root complexes. Thus, VH (virtual hierarchy) numbers need to be allocated to determine a virtual PCI express switch and a plurality of PCI-PCI bridges to be used by each of the root complexes. Since switch processing is executed per virtual PCI express switch, virtual PCI express switches do not interfere with each other.
Further, VH numbers are allocated to the MR-aware PCI-PCI bridges, and a plurality of PCI-PCI bridges are connected to a single link. Credit control or configuration information is managed separately based on different VH numbers, and a single port indicates a plurality of PCI-PCI bridge use numbers.
The MR-PCIM executes the allocation by setting the setting register storing control information about the MRA controller logic. FIGS. 4 and 5 illustrate configuration examples of the setting registers 5121 and 5131 of the MRA PCI express switches 511 and 512, respectively. Based on this configuration, the MR-PCIM is connected to a port 1 connected to the CPU 101 and the root complex 102. Since the MRA PCI express switches are set by the MR-PCIM, paths of the VH number 0 are used by the MR PCI-PCI bridges.
Next, the setting information stored in the setting registers will be described. Each of the setting registers stores virtual PCI express switch numbers and information forming each of the virtual PCI express switches. Namely, for each of the ports, the setting register stores a switch port use number, a use VH number, and UP/Down: DN information indicating an upstream PCI-PCI bridge or a downstream PCI-PCI bridge. In the setting register 5121, the virtual PCI express switch 1 5116 has a switch number 1 and is connected to: the port 1 (a VH number VH0 and an upstream PCI-PCI bridge UP); a port 3 (VH0 and DN); a port 4 (no VH number N/A treated as a normal PCIe port); and a port 6 (VH0 and DN). Namely, based on the setting information, the root complex 102 can access the MR-aware upstream PCI-PCI bridge 5111, the virtual PCI express switch 1 5116, the MR-aware downstream PCI-PCI bridge 5113, the MR end point 505, the downstream PCI-PCI bridge 5114, the end point 106, and the MR-aware downstream PCI-PCI bridge 5122.
Access beyond the MR-aware upstream/downstream PCI-PCI bridge 5122 can be made by setting the setting register 5131 of the MRA PCI express switch 512. Based on the setting register 5131, the virtual PCI express switch 2 5127 is connected to a port 2 (VH0 and DN) and the port 3 (VH0 and UP). Beyond the MR-aware upstream/downstream PCI-PCI bridge 5122, the root complex 102 can access the MR-aware upstream/downstream PCI-PCI bridge 5124, the virtual PCI express switch 2 5127, the MR-aware downstream PCI-PCI bridge 5133, and the MR end point 506.
Likewise, the virtual PCI express switch 2 5117 and the virtual PCI express switch 3 5118 of the setting register 5121 and the virtual PCI express switch 1 5126 of the setting register 5131 are set as illustrated in FIGS. 4 and 5. Based on these settings, PCI trees that include I/O devices and that are illustrated in FIGS. 6 and 7 are established for the CPUs 101 and 1011, respectively.    [Non-Patent Document I] Multi-Root I/O Virtualization and Sharing Specification Revision 0.9, PCI-SIG, Nov. 7, 2007, pp. 26-33